1. Field of the Invention
This invention relates to an image processing method and apparatus for carrying out image processing on an image signal. This invention also relates to an image composing method and system for composing an image from a plurality of images. This invention further relates to an image sharpness estimating method and device for estimating graininess of an image. This invention still further relates to an image sharpness estimating method and device for estimating sharpness of an image.
2. Description of the Prior Art
Techniques for obtaining an image signal, which represents an image, carrying out appropriate image processing on the image signal, and reproducing a visible image from the processed image signal have heretofore been carried out in various fields. An image output device for reproducing the image carries out image processing, such as correction of the image signal, in accordance with characteristics of an image input apparatus, which is connected to the image output device in a system, and characteristics of the image output device and reproduces the image from the processed image signal. In cases where only one image input apparatus is connected to the image output device, the image output device may carry out the image processing corresponding to the characteristics of the image input apparatus. In cases where a plurality of image input apparatuses are connected to the image output device, the image output device may carry out the image processing corresponding to the characteristics of each of the image input apparatuses.
However, in cases where a plurality of image input apparatuses are connected to an image output device, it often occurs that the image input apparatuses are altered, and it cannot be specified which image signal having been fed into the image output device is the one obtained from which image input apparatus. Therefore, it is difficult to carry out optimum image processing such that the details of the image processing may correspond to the characteristics of each of the image input apparatuses. Accordingly, an image processing apparatus has been proposed in, for example, Japanese Patent No. 2,660,170, wherein image signals having been received from image input apparatuses are converted into image signals having general-purpose gradation, and wherein only the output characteristics are corrected in an image output device such that an appropriate reproduced image may be obtained.
Also, images obtained from image input apparatuses have graininess characteristics (i.e., the state of noise components) and sharpness characteristics varying in accordance with the image input apparatuses. Therefore, ordinarily, image processing, such as smoothing processing or sharpness enhancement processing, is carried on the obtained image signal and in accordance with the graininess characteristics and/or the sharpness characteristics of the image input apparatus, from which the image signal has been obtained, such that an image having good image quality may be reproduced. Further, when an image is reproduced, image size enlargement or reduction is often carried out by carrying out an interpolating operation on an image signal. In such cases, an interpolating operation (e.g., a cubic spline interpolating operation), which attaches importance to sharpness, or an interpolating operation (e.g., a B spline interpolating operation), which attaches importance to smoothness, is carried out in accordance with the graininess characteristics and/or the sharpness characteristics of the image input apparatus, from which the image signal has been obtained.
As a technique for processing an image signal, a multi-resolution transforming technique has heretofore been proposed. With the multi-resolution transforming technique, an original image is transformed in to multi-resolution images, each of which is of one of a plurality of different frequency bands, and predetermined processing is carried out on the image of each frequency band. Inverse multi-resolution transform is then carried out, and an ultimate processed image is thereby obtained. In such cases, the predetermined processing may be processing for separating high frequency components for noise removal, compression processing in which signal components of a frequency band containing much noise are reduced, or the like. As the multi-resolution transforming technique, a wavelet transform technique, a Laplacian pyramid technique, a Fourier transform technique, or the like, has been known. In particular, the wavelet transform is one of techniques for signal frequency analysis. The wavelet transform is advantageous over the Fourier transform, which has heretofore been used widely as the frequency analysis technique, in that a local change in a signal can be detected easily. Therefore, recently, the wavelet transform has attracted particular attention in the field of signal processing (xe2x80x9cWavelets and Signal Processingxe2x80x9d by Olivier Rioul and Martin Vetterli, IEEE SP MAGAZINE, pp. 14-38, October 1991; xe2x80x9cZero-Crossings of a Wavelet Transformxe2x80x9d by Stephane Mallat, IEEE TRANSACTIONS ON INFORMATION THEORY, Vol. 37, No. 4, pp. 1,019-1,033, July 1991; Japanese Unexamined Patent Publication Nos. 6(1994)-350989, 6(1994)-350990, 7(1995)-23228, 7(1995)-23229, and 7(1995)-79350, and Japanese Patent Application No. 8(1996)-14510).
With rapid advances made in computer networks in recent years, various kinds of image input apparatuses are connected to an image output device, and image signals having been obtained from various kinds of processing are transferred through a network. Therefore, image signals are transferred in a state such that it may not be clear which image signal having been fed into the image output device is the one obtained from which image input apparatus, and such that it may not be clear whether image processing has or has not been carried out with respect to graininess, sharpness, or gradation of the image signal. In such cases, if the image processing apparatus proposed in Japanese Patent No. 2,660,170 is utilized, image processing can be carried out on image signals such that they may have general-purpose gradation regardless of the kinds of the image input apparatuses.
However, as for graininess (noise components) or sharpness of an image represented by an image signal, if it is not clear which image signal is the one having been obtained from which image input apparatus, image processing, such as smoothing processing, sharpness enhancement processing, or interpolating operation processing, which is optimum for the characteristics of the image input apparatus, cannot be carried out on the image signal. In such cases, it may be considered to carry out the same image processing with respect to graininess or sharpness on every image signal by utilizing one predetermined technique. However, the image processing cannot be optimum for every image, and there is the risk that, for example, an image has a high level of graininess, but the sharpness of the image is enhanced. Also, there is the risk that an unsharp image is obtained due to insufficient sharpness enhancement. Further, there is the risk that an image has a high level of sharpness, but the sharpness of the image is enhanced even further, resulting in an image having an artifact, such as overshooting or undershooting. Particularly, in cases where the technique described above is employed when an image is composed from a plurality of images and the composed image is reproduced, even if the image signals combined with one another have different graininess characteristics or different sharpness characteristics, the same image processing will be carried out on the image signals. Therefore, from the image composition, an image will be obtained, in which the balance of graininess or sharpness varies for different areas of the image and which gives an unnatural feeling. Also, it may be considered for the operator to view an image having been reproduced and to artificially correct the image signal. However, in such cases, considerable time and labor are required to make a correction, and the burden to the operator cannot be kept light.
The primary object of the present invention is to provide an image processing method, wherein image processing optimum for an image is capable of being carried out regardless of graininess and/or sharpness of the image.
Another object of the present invention is to provide an apparatus for carrying out the image processing method.
A further object of the present invention is to provide an image composing method, wherein image processing optimum for each of images, which are to be combined with one another, is capable of being carried out regardless of graininess and/or sharpness of each of the images.
A still further object of the present invention is to provide a system for carrying out the image composing method.
Another object of the present invention is to provide an image graininess estimating method, wherein graininess of an image is capable of being estimated from an image signal representing the image.
A further object of the present invention is to provide a device for carrying out the image graininess estimating method.
A still further object of the present invention is to provide an image sharpness estimating method, wherein sharpness of an image is capable of being estimated from an image signal representing the image.
Another object of the present invention is to provide a device for carrying out the image sharpness estimating method.
The present invention provides an image processing method, wherein image processing is carried out on an original image signal representing an original image, the method comprising the steps of:
i) estimating graininess and/or sharpness of the original image in accordance with the original image signal, and
ii) carrying out the image processing on the original image signal and in accordance with the graininess and/or the sharpness having been estimated, a processed image signal being thereby obtained.
In the image processing method in accordance with the present invention, the image processing should preferably be one of interpolating operation processes, which yield interpolation images having different levels of sharpness and each of which is set in accordance with the graininess and/or the sharpness having been estimated. Also, the image processing should preferably be one of sharpness enhancing processes, which enhance sharpness with different degrees of enhancement and each of which is set in accordance with the graininess and/or the sharpness having been estimated.
By way of example, the interpolating operation processes, which yield interpolation images having different levels of sharpness, may be a cubic spline interpolating operation process, which attaches importance to sharpness, and a B spline interpolating operation process, which attaches importance to smoothness.
The present invention also provides an image composing method, wherein a plurality of original image signals representing a plurality of images are combined with one another, and a composed image signal is thereby obtained, the method comprising the steps of:
i) carrying out image processing on each of the original image signals and with the image processing method in accordance with the present invention, a plurality of processed image signals being thereby obtained, and
ii) combining the plurality of the processed image signals, the composed image signal being thereby obtained.
The present invention further provides an image processing apparatus, wherein image processing is carried out on an original image signal representing an original image, the apparatus comprising:
i) an estimating means for estimating graininess and/or sharpness of the original image in accordance with the original image signal, and
ii) an image processing means for carrying out the image processing on the original image signal and in accordance with the graininess and/or the sharpness having been estimated, and thereby obtaining a processed image signal.
In the image processing apparatus in accordance with the present invention, the image processing, which is carried out by the image processing means, should preferably be one of interpolating operation processes, which yield interpolation images having different levels of sharpness and each of which is set in accordance with the graininess and/or the sharpness having been estimated. Also, the image processing, which is carried out by the image processing means, should preferably be one of sharpness enhancing processes, which enhance sharpness with different degrees of enhancement and each of which is set in accordance with the graininess and/or the sharpness having been estimated.
The present invention still further provides an image composing system, wherein a plurality of original image signals representing a plurality of images are combined with one another, and a composed image signal is thereby obtained, the system comprising:
i) means for carrying out image processing on each of the original image signals and with the image processing apparatus in accordance with the present invention, and thereby obtaining a plurality of processed image signals, and
ii) an image composing means for combining the plurality of the processed image signals, and thereby obtaining the composed image signal.
The present invention also provides an image graininess estimating method, comprising the steps of:
i) transforming an original image signal, which represents an original image, into a multi-resolution space, the original image signal being thereby decomposed into image signals representing images, each of which is of one of a plurality of different frequency bands,
ii) comparing a pixel value of each of pixels in a low frequency band image and a predetermined threshold value with each other, the low frequency band image being of a frequency band lower than the frequency band of the highest frequency band image, which is of the highest frequency band, among the images of the plurality of the different frequency bands,
iii) calculating a variance value of pixel values of pixels in the highest frequency band image, which correspond to the pixels in the low frequency band image that have been found to have values smaller than the predetermined threshold value, and
iv) estimating graininess of the original image in accordance with the variance value.
The term xe2x80x9clow frequency band imagexe2x80x9d as used herein means the image of the frequency band lower by one stage than the frequency band of the highest frequency band image, or the image of the frequency band lower by at least two stages than the frequency band of the highest frequency band image.
The term xe2x80x9cin accordance with a variance valuexe2x80x9d as used herein means that the estimation is made in accordance with the level of the variance value.
In the image graininess estimating method in accordance with the present invention, the transform into the multi-resolution space, the comparison with the predetermined threshold value, and the calculation of the variance value should preferably be carried out with respect to only a region selected in the original image.
The selected region may be a center region of the image, which region has a high possibility of containing a major object pattern, or may be an arbitrary region in the image.
Also, in the image graininess estimating method in accordance with the present invention, the transform into the multi-resolution space should preferably be carried out with wavelet transform. Alternatively, the transform into the multi-resolution space may be carried out with one of other multi-resolution transform techniques, such as the Laplacian pyramid technique and the Fourier transform technique.
Ordinarily, in the multi-resolution transform techniques, such as the wavelet transform, pixels are thinned out each time the transform is carried out, and therefore an image having a size reduced from the size of the original image is obtained as the frequency band becomes low. However, in the image graininess estimating method in accordance with the present invention, the transform into the multi-resolution space should preferably be carried out such that the pixels in the images of the plurality of the different frequency bands may not be thinned out.
The present invention further provides an image graininess estimating device, comprising:
i) a multi-resolution transforming means for transforming an original image signal, which represents an original image, into a multi-resolution space, and thereby decomposing the original image signal into image signals representing images, each of which is of one of a plurality of different frequency bands,
ii) a comparison means for comparing a pixel value of each of pixels in a low frequency band image and a predetermined threshold value with each other, the low frequency band image being of a frequency band lower than the frequency band of the highest frequency band image, which is of the highest frequency band, among the images of the plurality of the different frequency bands,
iii) a variance value calculating means for calculating a variance value of pixel values of pixels in the highest frequency band image, which correspond to the pixels in the low frequency band image that have been found to have values smaller than the predetermined threshold value, and
iv) an estimating means for estimating graininess of the original image in accordance with the variance value.
In the image graininess estimating device in accordance with the present invention, the multi-resolution transforming means, the comparison means, and the variance value calculating means, respectively, should preferably carry out the transform into the multi-resolution space, the comparison with the predetermined threshold value, and the calculation of the variance value with respect to only a region selected in the original image.
Also, in the image graininess estimating device in accordance with the present invention, the multi-resolution transforming means should preferably carry out the transform into the multi-resolution space with wavelet transform.
Further, in the image graininess estimating device in accordance with the present invention, the multi-resolution transforming means should preferably carry out the transform into the multi-resolution space such that the pixels in the images of the plurality of the different frequency bands may not be thinned out.
The present invention still further provides an image sharpness estimating method, comprising the steps of:
i) transforming an original image signal, which represents an original image, into a multi-resolution space, the original image signal being thereby decomposed into image signals representing images, each of which is of one of a plurality of different frequency bands,
ii) comparing a pixel value of each of pixels in a low frequency band image and a predetermined threshold value with each other, the low frequency band image being of a frequency band lower than the frequency band of the highest frequency band image, which is of the highest frequency band, among the images of the plurality of the different frequency bands,
iii) calculating a ratio between a pixel value of a pixel in the low frequency band image, which pixel has been found to have a value larger than the predetermined threshold value, and the pixel value of the corresponding pixel in the highest frequency band image, and
iv) estimating sharpness of the original image in accordance with the ratio.
In the image sharpness estimating method in accordance with the present invention, the low frequency band image may be the image of the frequency band lower by one stage than the frequency band of the highest frequency band image, or the image of the frequency band lower by at least two stages than the frequency band of the highest frequency band image.
The term xe2x80x9cin accordance with a ratioxe2x80x9d as used herein means that the estimation is made in accordance with the level of the ratio.
The image sharpness estimating method in accordance with the present invention should preferably be modified such that the transform into the multi-resolution space may be carried out by:
carrying out filtering processes, respectively, on the original image along each of a vertical direction and a horizontal direction, a vertical-direction high frequency/horizontal-direction low frequency image, a vertical-direction low frequency/horizontal-direction high frequency image, and a vertical- and horizontal-direction low frequency image being thereby obtained,
carrying out filtering processes on the vertical- and horizontal-direction low frequency image, and
repeating filtering processes successively on the vertical- and horizontal-direction low frequency images, which have been obtained from the previous filtering processes, vertical-direction high frequency/horizontal-direction low frequency images and vertical-direction low frequency/horizontal-direction high frequency images being thereby obtained, each of the vertical-direction high frequency/horizontal-direction low frequency images and each of the vertical-direction low frequency/horizontal-direction high frequency images being of one of the plurality of the different frequency bands,
the comparison with the predetermined threshold value may be carried out with respect to each of the vertical-direction high frequency/horizontal-direction low frequency image and the vertical-direction low frequency/horizontal-direction high frequency image, which are of the low frequency band, and
the ratio may be the ratio between a pixel value of a pixel in the vertical-direction high frequency/horizontal-direction low frequency image of the low frequency band, which pixel has been found to have a value larger than the predetermined threshold value, and the pixel value of the corresponding pixel in the highest frequency band image, or the ratio between a pixel value of a pixel in the vertical-direction low frequency/horizontal-direction high frequency image of the low frequency band, which pixel has been found to have a value larger than the predetermined threshold value, and the pixel value of the corresponding pixel in the highest frequency band image, whichever ratio has a larger value.
The term xe2x80x9crepeating filtering processes successively on vertical- and horizontal-direction low frequency images having been obtained from previous filtering processesxe2x80x9d as used herein means that the filtering processes are carried out on the vertical- and horizontal-direction low frequency image, which has been obtained each time the previous filtering processes have been carried out.
In the image sharpness estimating method in accordance with the present invention, the transform into the multi-resolution space, the comparison with the predetermined threshold value, and the calculation of the ratio should preferably be carried out with respect to only a region selected in the original image.
The selected region may be a center region of the image, which region has a high possibility of containing a major object pattern, or may be an arbitrary region in the image.
Also, in the image sharpness estimating method in accordance with the present invention, the transform into the multi-resolution space should preferably be carried out with wavelet transform. Alternatively, the transform into the multi-resolution space may be carried out with one of other multi-resolution transform techniques, such as the Laplacian pyramid technique and the Fourier transform technique.
Ordinarily, in the multi-resolution transform techniques, such as the wavelet transform, pixels are thinned out each time the transform is carried out, and therefore an image having a size reduced from the size of the original image is obtained as the frequency band becomes low. However, in the image sharpness estimating method in accordance with the present invention, the transform into the multi-resolution space should preferably be carried out such that the pixels in the images of the plurality of the different frequency bands may not be thinned out.
The present invention also provides an image sharpness estimating device, comprising:
i) a multi-resolution transforming means for transforming an original image signal, which represents an original image, into a multi-resolution space, the original image signal being thereby decomposed into image signals representing images, each of which is of one of a plurality of different frequency bands,
ii) a comparison means for comparing a pixel value of each of pixels in a low frequency band image and a predetermined threshold value with each other, the low frequency band image being of a frequency band lower than the frequency band of the highest frequency band image, which is of the highest frequency band, among the images of the plurality of the different frequency bands,
iii) a ratio calculating means for calculating a ratio between a pixel value of a pixel in the low frequency band image, which pixel has been found to have a value larger than the predetermined threshold value, and the pixel value of the corresponding pixel in the highest frequency band image, and
iv) an estimating means for estimating sharpness of the original image in accordance with the ratio.
The image sharpness estimating device in accordance with the present invention should preferably be modified such that the multi-resolution transforming means may carry out the transform into the multi-resolution space by:
carrying out filtering processes, respectively, on the original image along each of a vertical direction and a horizontal direction, a vertical-direction high frequency/horizontal-direction low frequency image, a vertical-direction low frequency/horizontal-direction high frequency image, and a vertical- and horizontal-direction low frequency image being thereby obtained,
carrying out filtering processes on the vertical- and horizontal-direction low frequency image, and
repeating filtering processes successively on the vertical- and horizontal-direction low frequency images, which have been obtained from the previous filtering processes, vertical-direction high frequency/horizontal-direction low frequency images and vertical-direction low frequency/horizontal-direction high frequency images being thereby obtained, each of the vertical-direction high frequency/horizontal-direction low frequency images and each of the vertical-direction low frequency/horizontal-direction high frequency images being of one of the plurality of the different frequency bands,
the comparison means may carry out the comparison with the predetermined threshold value with respect to each of the vertical-direction high frequency/horizontal-direction low frequency image and the vertical-direction low frequency/horizontal-direction high frequency image, which are of the low frequency band, and
the ratio calculating means may calculate, as the ratio, the ratio between a pixel value of a pixel in the vertical-direction high frequency/horizontal-direction low frequency image of the low frequency band, which pixel has been found to have a value larger than the predetermined threshold value, and the pixel value of the corresponding pixel in the highest frequency band image, or the ratio between a pixel value of a pixel in the vertical-direction low frequency/horizontal-direction high frequency image of the low frequency band, which pixel has been found to have a value larger than the predetermined threshold value, and the pixel value of the corresponding pixel in the highest frequency band image, whichever ratio has a larger value.
In the image sharpness estimating device in accordance with the present invention, the multi-resolution transforming means, the comparison means, and the ratio calculating means, respectively, should preferably carry out the transform into the multi-resolution space, the comparison with the predetermined threshold value, and the calculation of the ratio with respect to only a region selected in the original image.
Also, in the image sharpness estimating device in accordance with the present invention, the multi-resolution transforming means should preferably carry out the transform into the multi-resolution space with wavelet transform.
Further, in the image sharpness estimating device in accordance with the present invention, the multi-resolution transforming means should preferably carry out the transform into the multi-resolution space such that the pixels in the images of the plurality of the different frequency bands may not be thinned out.
With the image processing method and apparatus in accordance with the present invention, graininess and/or sharpness of the original image is estimated in accordance with the original image signal, and the image processing is carried out on the original image signal and in accordance with the graininess and/or the sharpness having been estimated. Therefore, by the utilization of the results of estimation of graininess and/or sharpness, a smoothing process, a sharpness enhancing process, an interpolating operation process, or the like, can be carried out on the original image signal and in accordance with the graininess and/or the sharpness of the original image. For example, in cases where it has been estimated that the level of graininess is high, a smoothing process for reducing the graininess can be carried out on the original image signal or, when an interpolating operation is to be carried out, the B spline interpolating operation process, which attaches importance to smoothness, can be carried out on the original image signal. In cases where it has been estimated that the level of graininess is low, no smoothing process may be carried out on the original image signal or, when an interpolating operation is to be carried out, the cubic spline interpolating operation process, which attaches importance to sharpness, can be carried out on the original image signal. Also, a sharpness enhancing process can be carried out such that the sharpness may be less enhanced at an area in the original image, which has been estimated as having a high level of sharpness, and such that the sharpness may be more enhanced at an area in the original image, which has been estimated as having a low level of sharpness. Further, in cases where it has been estimated that the level of sharpness is high in the entire area of the original image and image size enlargement or reduction is to be carried out, the B spline interpolating operation process, which attaches importance to smoothness, can be carried out on the original image signal. In cases where it has been estimated that the level of sharpness is low in the entire area of the original image and image size enlargement or reduction is to be carried out, the cubic spline interpolating operation process, which attaches importance to sharpness, can be carried out on the original image signal. Accordingly, even if it is not clear which image signal representing an original image is the one having been obtained from which image input apparatus, and even if it is not clear whether the received image signal has or has not been subjected to image processing, the image processing with respect to graininess and/or sharpness, which image processing is optimum for the original image, can be carried out.
With the image composing method and system in accordance with the present invention, the image processing is carried out on each of a plurality of original image signals and with the image processing method and apparatus in accordance with the present invention, and a plurality of processed image signals are thereby obtained. The plurality of the processed image signals are then combined with one another, and the composed image signal is thereby obtained. Therefore, the composed image signal can be obtained from the processed image signals, each of which has been obtained from the optimum image processing having been carried out in accordance with the graininess and/or the sharpness of the original image. Accordingly, the problems can be prevented from occurring in that, in cases where the same image processing is carried out on all of the original image signals to be combined, an image is obtained, in which the balance of graininess or sharpness varies for different areas of the image. As a result, a composed image can be obtained, which gives a natural feeling.
With the image graininess estimating method and device in accordance with the present invention, the original image signal is firstly transformed into a multi-resolution space by utilizing the wavelet transform, or the like, and the original image signal is thereby decomposed into image signals representing images, each of which is of one of a plurality of different frequency bands. Thereafter, the pixel value of each of pixels in the low frequency band image and the predetermined threshold value are compared with each other, the low frequency band image being of a frequency band lower than the frequency band of the highest frequency band image among the images of the plurality of the different frequency bands. A boundary line, such as an edge, which is embedded in the original image, has a large pixel value. The pixel value of the boundary line is large also in the highest frequency band image and the low frequency band image. Also, as in the boundary line, a grainy component (a noise component) contained in the original image has a large pixel value. The pixel value of the grainy component is large also in the highest frequency band image. However, in the low frequency band image, the grainy component, which constitutes a high frequency component of the original image, has been removed, and therefore the pixel value of the pixel corresponding to the grainy component is comparatively small. Therefore, in accordance with the results of comparison made between the pixel value of each of pixels in the low frequency band image and the predetermined threshold value, the pixels in the highest frequency band image, which correspond to the pixels in the low frequency band image that have been found to have values smaller than the predetermined threshold value, are regarded as the pixels having a possibility of containing grainy components. With respect to such pixels in the highest frequency band image, the variance value of the pixel values is calculated. In cases where grainy components are contained in a flat image area (i.e., an image area at which the signal values are approximately identical with one another), the variance value becomes large. In cases where little grainy component is contained in the flat image area, the variance value becomes small. Accordingly, the graininess of the original image can be estimated in accordance with the level of the variance value.
As described above, with the image graininess estimating method and device in accordance with the present invention, the graininess of the original image can be estimated from the original image signal representing the original image. Therefore, in an image processing apparatus, by the utilization of the results of estimation, a smoothing process, an interpolating operation process, or the like, in accordance with the estimated graininess can be carried out on the original image signal. For example, in cases where it has been estimated that the level of graininess is high, a smoothing process for reducing the graininess can be carried out on the original image signal or, when an interpolating operation is to be carried out, the B spline interpolating operation process, which attaches importance to smoothness, can be carried out on the original image signal. In cases where it has been estimated that the level of graininess is low, no smoothing process may be carried out on the original image signal or, when an interpolating operation is to be carried out, the cubic spline interpolating operation process, which attaches importance to sharpness, can be carried out on the original image signal. Accordingly, even if it is not clear which image signal representing an original image is the one having been obtained from which image input apparatus, and even if it is not clear whether the received image signal has or has not been subjected to image processing, the image processing with respect to graininess, which image processing is optimum for the original image, can be carried out.
Also, with the image graininess estimating method and device in accordance with the present invention, the transform into the multi-resolution space, the comparison with the predetermined threshold value, and the calculation of the variance value may be carried out with respect to only a region selected in the original image. In such cases, the amount of calculations can be kept small, and the estimation of graininess can be carried out quickly.
Further, with the image graininess estimating method and device in accordance with the present invention, the transform into the multi-resolution space may be carried out such that the pixels in the images of the plurality of the different frequency bands may not be thinned out. In such cases, the correspondence between the pixels in the low frequency band image and the pixels in the highest frequency band image can be set accurately.
With the image sharpness estimating method and device in accordance with the present invention, the original image signal is firstly transformed into a multi-resolution space by utilizing the wavelet transform, or the like, and the original image signal is thereby decomposed into image signals representing images, each of which is of one of a plurality of different frequency bands. Thereafter, the pixel value of each of pixels in the low frequency band image and the predetermined threshold value are compared with each other, the low frequency band image being of a frequency band lower than the frequency band of the highest frequency band image among the images of the plurality of the different frequency bands. An image area, such as an edge embedded in the original image, which serves as a reference in the estimation of sharpness of the original image, has a comparatively large pixel value. The pixel value of such an image area is large also in the low frequency band image. Conversely, if the pixel value of a pixel in the low frequency band image is small, it can be considered that an image area, such as an edge, which serves as a reference in the estimation of sharpness, is not present at the pixel. Therefore, in accordance with the results of comparison made between the pixel value of each of pixels in the low frequency band image and the predetermined threshold value, the pixel in the low frequency band image, which pixel has been found to have a value larger than the predetermined threshold value, is regarded as being a pixel serving as a reference in the estimation of sharpness. A calculation is made to find the ratio between the pixel value of the pixel in the low frequency band image, which pixel has been found to have a value larger than the predetermined threshold value, and the pixel value of the corresponding pixel in the highest frequency band image.
The highest frequency band image carries the highest frequency components of the original image, and therefore represents an amount of change in signal value in a comparatively narrow range. The low frequency band image carries the low frequency components of the original image, and therefore represents an amount of change in signal value in a range wider than in the highest frequency band image. For example, as illustrated in FIG. 5A, at an edge at which the signal value changes sharply, an amount of change in signal value xcex94A in a narrow range A and an amount of change in signal value xcex94B in a wide range B are identical with each other. Also, as illustrated in FIG. 5B, at an image area at which the change in signal value is not sharp, the frequency components are of a frequency band lower than that at the edge at which the signal value changes sharply, and therefore an amount of change in signal value xcex94B in a wide range B becomes larger than an amount of change in signal value xcex94A in a narrow range A. Accordingly, at the edge at which the signal value changes sharply as illustrated in FIG. 5A, the pixel corresponding to the edge area takes an approximately identical pixel value in the highest frequency band image and the low frequency band image. Also, at the image area at which the change in signal value is not sharp as illustrated in FIG. 5B, the pixel corresponding to the signal change position takes a larger pixel value in the low frequency band image than in the highest frequency band image.
Therefore, the ratio between the pixel value of the pixel in the low frequency band image and the pixel value of the corresponding pixel in the highest frequency band image takes a value close to 1 at the edge at which the signal value changes sharply, i.e. at an image area having a high level of sharpness. Also, the ratio takes a value deviated from 1 at an image area at which the change in signal value is not sharp, i.e. at an image area having a low level of sharpness. Accordingly, the sharpness of the original image can be estimated in accordance with the calculated value of the ratio.
As described above, with the image sharpness estimating method and device in accordance with the present invention, the sharpness of the original image can be estimated from the original image signal representing the original image. Therefore, in an image processing apparatus, by the utilization of the results of estimation, a sharpness enhancing process, an interpolating operation process, or the like, in accordance with the estimated sharpness can be carried out on the original image signal. For example, a sharpness enhancing process can be carried out such that the sharpness may be less enhanced at an area in the original image, which has been estimated as having a high level of sharpness, and such that the sharpness may be more enhanced at an area in the original image, which has been estimated as having a low level of sharpness. Further, in cases where it has been estimated that the level of sharpness is high in the entire area of the original image and image size enlargement or reduction is to be carried out, the B spline interpolating operation process, which attaches importance to smoothness, can be carried out on the original image signal. In cases where it has been estimated that the level of sharpness is low in the entire area of the original image and image size enlargement or reduction is to be carried out, the cubic spline interpolating operation process, which attaches importance to sharpness, can be carried out on the original image signal. Accordingly, even if it is not clear which image signal representing an original image is the one having been obtained from which image input apparatus, and even if it is not clear whether the received image signal has or has not been subjected to image processing, the image processing with respect to sharpness, which image processing is optimum for the original image, can be carried out.
With the image sharpness estimating method and device in accordance with the present invention, the transform into the multi-resolution space may be carried out by repeating the filtering processes along each of the vertical direction and the horizontal direction in the original image. Also, the comparison between the pixel value of each pixel and the predetermined threshold value may be carried out with respect to each of the vertical-direction high frequency/horizontal-direction low frequency image and the vertical-direction low frequency/horizontal-direction high frequency image, which are of the low frequency band. Further, a calculation may be made to find, as the ratio for the estimation of sharpness, the ratio between the pixel value of the pixel in the vertical-direction high frequency/horizontal-direction low frequency image of the low frequency band, which pixel has been found to have a value larger than the predetermined threshold value, and the pixel value of the corresponding pixel in the highest frequency band image, or the ratio between the pixel value of the pixel in the vertical-direction low frequency/horizontal-direction high frequency image of the low frequency band, which pixel has been found to have a value larger than the predetermined threshold value, and the pixel value of the corresponding pixel in the highest frequency band image, whichever ratio has a larger value. In such cases, the sharpness can be estimated without depending upon the directivity of the edge, or the like, contained in the original image.
Also, with the image sharpness estimating method and device in accordance with the present invention, the transform into the multi-resolution space, the comparison with the predetermined threshold value, and the calculation of the ratio may be carried out with respect to only a region selected in the original image. In such cases, the amount of calculations can be kept small, and the estimation of sharpness can be carried out quickly.
Further, with the image sharpness estimating method and device in accordance with the present invention, the transform into the multi-resolution space may be carried out such that the pixels in the images of the plurality of the different frequency bands may not be thinned out. In such cases, the correspondence between the pixels in the low frequency band image and the pixels in the highest frequency band image can be set accurately.